Fixed-delay Events in Generalized 
Semi-Markov Processes Revisited* 



Tomas Brazdil, Jan Krcal, Jan Kfetmsky**, and Vojtech Rehak 

Faculty of Informatics, Masaryk University, Brno, Czech Republic 
{brazdil, krcal, jan.kretinsky, rehak} @fi. muni.cz 



Abstract. We study long run average behavior of generalized semi-Markov pro- 
cesses with both fixed-delay events as well as variable-delay events. We show that 
allowing two fixed-delay events and one variable-delay event may cause an unsta- 
ble behavior of a GSMP. In particular, we show that a frequency of a given state 
may not be defined for almost all runs (or more generally, an invariant measure 
may not exist). We use this observation to disprove several results from litera- 
ture. Next we study GSMP with at most one fixed-delay event combined with an 
arbitrary number of variable-delay events. We prove that such a GSMP always 
possesses an invariant measure which means that the frequencies of states are 
always well defined and we provide algorithms for approximation of these fre- 
quencies. Additionally, we show that the positive results remain valid even if we 
allow an arbitrary number of reasonably restricted fixed-delay events. 



1 Introduction 

Generalized semi-Markov processes (GSMP), introduced by Matthes in ll22l . are a stan- 
dard model for discrete-event stochastic systems. Such a system operates in continuous 
time and reacts, by changing its state, to occurrences of events. Each event is assigned a 
random delay after which it occurs; state transitions may be randomized as well. When- 
ever the system reacts to an event, new events may be scheduled and pending events may 
be discarded. To get some intuition, imagine a simple communication model in which a 
server sends messages to several clients asking them to reply. The reaction of each client 
may be randomly delayed, e.g., due to latency of communication links. Whenever a re- 
ply comes from a client, the server changes its state (e.g., by updating its database of 
alive clients or by sending another message to the client) and then waits for the rest of 
the replies. Such a model is usually extended by allowing the server to time-out and to 
take an appropriate action, e.g., demand replies from the remaining clients in a more 
urgent way. The time-out can be seen as another event which has a fixed delay. 

More formally, a GSMP consists of a set S of states and a set S of events. Each 
state s is assigned a set E(s) of events scheduled in s. Intuitively, each event in E(s) is 
assigned a positive real number representing the amount of time which elapses before 
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the event occurs. Note that several events may occur at the same time. Once a set of 
events E c E(s) occurs, the system makes a transition to a new state s' . The state s' 
is randomly chosen according to a fixed distribution which depends only on the state s 
and the set E. In s', the old events of E(s) \ E(Y ) are discarded, each inherited event of 
(E(i') | ~ | E(i)) \E remains scheduled to the same point in the future, and each new event 
of (E(s') \ E(*)) U (E(s') n E) is newly scheduled according to its given probability 
distribution. 

In order to deal with GSMP in a rigorous way, one has to impose some restrictions 
on the distributions of delays. Standard mathematical literature, such as 0141151 . usually 
considers GSMP with continuously distributed delays. This is certainly a limitation, as 
some systems with fixed time delays (such as time-outs or processor ticks) cannot be 
faithfully modeled using only continuously distributed delays. We show some exam- 
ples where fixed delays exhibit qualitatively different behavior than any continuously 
distributed approximation. In this paper we consider the following two types of events: 

- variable-delay: the delay of the event is randomly distributed according to a proba- 
bility density function which is continuous and positive either on a bounded interval 
[£, u] or on an unbounded interval [{, oo); 

- fixed-delay: the delay is set to a fixed value with probability one. 

The desired behavior of systems modeled using GSMP can be specified by various 
means. One is often interested in long-run behavior such as mean response time, fre- 
quency of errors, etc. (see, e.g., 0]). For example, in the above communication model, 
one may be interested in average response time of clients or in average time in which 
all clients eventually reply. Several model independent formalisms have been devised 
for expressing such properties of continuous time systems. For example, a well known 
temporal logic CSL contains a steady state operator expressing frequency of states 
satisfying a given subformula. In [9|, we proposed to specify long-run behavior of a 
continuous-time process using a timed automaton which observes runs of the process, 
and measure the frequency of locations of the automaton. 

In this paper we consider a standard performance measure, the frequency of states 
of the GSMP. To be more specific, let us fix a state s e S . We define a random variable 
d which to every run assigns the (discrete) frequency of visits to s on the run, i.e. the 
ratio of the number of transitions entering s to the number of all transitions. We also 
define a random variable c which gives timed frequency of s, i.e. the ratio of the amount 
of time spent in s to the amount of time spent in all states. Technically, both variables 
d and c are defined as limits of the corresponding ratios on prefixes of the run that 
are prolonged ad infinitum. Note that the limits may not be defined for some runs. For 
example, consider a run which alternates between s and another state s; it spends 2 time 
unit in s, then 4 in s, then 8 in s, then 16 in s, etc. Such a run does not have a limit 
ratio between time spent in s and in s. We say that d (or c) is well-defined for a run if 
the limit ratios exist for this run. Our goal is to characterize stable systems that have 
the variables d and c well-defined for almost all runs, and to analyze the probability 
distributions of d and c on these stable systems. 

As a working example of GSMP with fixed-delay events, we present a simplified 
protocol for time synchronization. Using the variable c, we show how to measure relia- 
bility of the protocol. Via message exchange, the protocol sets and keeps a client clock 
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Fig. 1. A GSMP model of a clock synchronization protocol. Below each state label, we list the 
set of scheduled events. We only display transitions that can take place with non-zero probability. 

sufficiently close to a server clock. Each message exchange is initialized by the client 
asking the server for the current time, i.e. sending a query message. The server adds a 
timestamp into the message and sends it back as a response. This query-response ex- 
change provides a reliable data for synchronization action if it is realized within a given 
round-trip delay. Otherwise, the client has to repeat the procedure. After a success, the 
client is considered to be synchronized until a given stable-time delay elapses. Since the 
aim is to keep the clocks synchronized all the time, the client restarts the synchroniza- 
tion process sooner, i.e. after a given polling delay that is shorter than the stable-time 
delay. Notice that the client gets desynchronized whenever several unsuccessful syn- 
chronizations occur in a row. Our goal is to measure the portion of the time when the 
client clock is not synchronized. 

Figure Q] shows a GSMP model of this protocol. The delays specified in the proto- 
col are modeled using fixed-delay events roundtrip_d, stable_d, and polling_d while 
actions are modeled by variable-delay events query, response, and sync. Note that if 
the stable-time runs out before a fast enough response arrives, the systems moves into 
primed states denoting it is not synchronized at the moment. Thus, c(Init') + c(Q-sent') 
expresses the portion of the time when the client clock is not synchronized. 

Our contribution. So far, GSMP were mostly studied with variable-delay events only. 
There are a few exceptions such as Q413I8I2I but they often contain erroneous statements 
due to presence of fixed-delay events. Our goal is to study the effect of mixing a number 
of fixed-delay events with an arbitrary amount of variable-delay events. 

At the beginning we give an example of a GSMP with two fixed-delay events for 
which it is not true that the variables d and c are well-defined for almost all runs. We 
also disprove some crucial statements of [3.4 1. In particular, we show an example of 
a GSMP which reaches one of its states with probability less than one even though 
the algorithms of II 3 141 return the probability one. The mistake of these algorithms is 
fundamental as they neglect the possibility of unstable behavior of GSMP. 

Concerning positive results, we show that if there is at most one fixed-delay event, 
then both d and c are almost surely well-defined. This is true even if we allow an arbi- 
trary number of reasonably restricted fixed-delay events. We also show how to approxi- 
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mate distribution functions of d and c. To be more specific, we show that for GSMP with 
at most one unrestricted and an arbitrary number of restricted fixed-delay events, both 
variables d and c have finite ranges {d\, ...,£/„} and {ci, . . . , c„). Moreover, all values d, 
and ci and probabilities P(d = di) and P(c = c,) can be effectively approximated. 

Related work. There are two main approaches to the analysis of GSMP. One is to re- 
strict the amount of events or types of their distributions and to solve the problems using 
symbolic methods [8 2 20]. The other is to estimate the values of interest using simu- 
lation B26I14I15L Concerning the first approach, time -bounded reachability has been 
studied in [2] where the authors restricted the delays of events to so called expolyno- 
mial distributions. The same authors also studied reachability probabilities of GSMP 
where in each transition at most one event is inherited [8 |. Further, the widely studied 
formalisms of semi-Markov processes (see, e.g., 019191 ) and continuous-time Markov 
chains (see, e.g., M6I7II ) are both subclasses of GSMP. 

As for the second approach, GSMP are studied by mathematicians as a stan- 
dard model for discrete event simulation and Markov chains Monte Carlo (see, e.g., 
[13 16 24|). Our work is strongly related to [1141151 where the long-run average behavior 
of GSMP with variable-delay events is studied. Under relatively standard assumptions 
the stochastic process generated by a GSMP is shown to be irreducible and to possess an 
invariant measure. In such a case, the variables d and c are almost surely constant. Be- 
side the theoretical results, there exist tools that employ simulation for model checking 
(see, e.g., B26I10I0 . 

In addition, GSMP are a proper subset of stochastic automata, a model of concur- 
rent systems (see, e.g., ifTTI '). Further, as shown in [15], GSMP have the same modeling 
power as stochastic Petri nets [21]. The formalism of deterministic and stochastic Petri 
nets (DSPN) introduced by [20 1 adds deterministic transitions - a counterpart of fixed- 
delay events. The authors restricted the model to at most one deterministic transition 
enabled at a time and to exponentially distributed timed transitions. For this restricted 
model, the authors proved existence of a steady state distribution and provided an al- 
gorithm for its computation. However, the methods inherently rely on the properties of 
the exponential distribution and cannot be extended to our setting with general variable 
delays. DSPN have been extended by 11211811 to allow arbitrarily many deterministic 
transitions. The authors provide algorithms for steady-state analysis of DSPN that were 
implemented in the tool DSPNExpress [17], but do not discuss under which conditions 
the steady-state distributions exist. 

2 Preliminaries 

In this paper, the sets of all positive integers, non-negative integers, real numbers, pos- 
itive real numbers, and non-negative real numbers are denoted by N, No, R, R>o, and 
R>o, respectively. For a real number r € R, int(r) denotes its integral part, i.e. the largest 
integer smaller than r, and frac(r) denotes its fractional part, i.e. r - int(r). Let A be a 
finite or countably infinite set. A probability distribution on A is a function / : A — > R>o 
such that YjaeA f( a ) - 1- The set of all distributions on A is denoted by D(A). 
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A cr-field over a set Q is a set T c 2 that includes and is closed under comple- 
ment and countable union. A measurable space is a pair (Q, T) where Q is a set called 
sample space and IF is a cr-field over whose elements are called measurable sets. 
Given a measurable space (Q, T), we say that a function / : Q — > R is a random vari- 
able if the inverse image of any real interval is a measurable set. A probability measure 
over a measurable space (Q, T) is a function P : T — > R>o such that, for each countable 
collection {X ; } ;e/ of pairwise disjoint elements of T , we have r P(\Jj e iX i ) = 2ie/^ , (^i) 
and, moreover, !P(.Q) = 1. A probability space is a triple (£?, f, P), where (Q,T) is a 
measurable space and P is a probability measure over (Q, T). We say that a property 
AcQ holds for almost all elements of a measurable set Y if !P(T) > 0, A n 7 e f , and 
P(A n y | Y) = 1. Alternatively, we say that A holds almost surely for F. 

2.1 Generalized semi-Markov processes 

Let £ be a finite set of events. To every e e & we associate the iower bound l e e No 
and the upper bound u e e N U {oo} of its delay. We say that e is a. fixed-delay event 
if £ e = M e , and a variable-delay event if £ e < M e . Furthermore, we say that a variable- 
delay event e is bounded if m<, ^ oo, and unbounded, otherwise. To each variable-delay 
event e we assign a density function f e : R — > R such that ' f e (x) dx — 1. We assume 
/ c to be positive and continuous on the whole \t e , u e ] or \t e , oo) if e is bounded or 
unbounded, respectively, and zero elsewhere. We require that f e have finite expected 
value, i.e. J ( ' x ■ f e (x) dx < oo. 

Definition 1. A generalized semi-Markov process is a tuple (S,S, E, Succ, ao) where 

- S is a finite set of states, 

- £ is a finite set of events, 

- E : S — > 2 £ assigns to each state s a set of events E(s) + scheduled to occur in s, 

- Succ : S x 2 s — > is ?/ze successor function, i.e. assigns a probability dis- 
tribution specifying the successor state to each state and set of events that occur 
simultaneously in this state, and 

- ao € D{S ) is the initial distribution. 

A configuration is a pair {s, v) where s e S and v is a valuation which assigns to 
every event e e E(s) the amount of time that elapsed since the event e was scheduled^ 
For convenience, we define v(e) = ± whenever e t E(s), and we denote by v(a) the 
amount of time spent in the previous configuration (initially, we put v(a) = 0). When 
a set of events E occurs and the process moves from s to a state s', the valuation of 
old events of E(s) \ E(s') is discarded to ±, the valuation of each inherited event of 
(E(Y) n E(s)) \ E is increased by the time spent in s, and the valuation of each new 
event of (E(s') \ E(s)) U (E(s') n E) is set to 0. 

We illustrate the dynamics of GSMP on the example of Figure Q] Let 
the bounds of the fixed-delay events roundtrip_d, polling_d, and stable_d be 

1 Usually, the valuation is defined to store the time left before the event appears. However, our 
definition is equivalent and more convenient for the general setting where both bounded and 
unbounded events appear. 
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1, 90, and 100, respectively. We start in the state Idle, i.e. in the configu- 
ration {Idle, ((polling_d, 0), (stable_d, 0), (A, 0))) denoting that v(polling_d) = 0, 
v(stable_d) = 0, v(a) = 0, and ± is assigned to all other events. After 90 time units, 
the event polling_d occurs and we move to (Jnit, {{query, 0), (stable_d, 90), (a, 90))). 
Assume that the event query occurs in the state Init after 0.6 time units and we move to 
(Q-sent, ((response, 0), (roundtrip_d, 0), (stable_d, 90.6), (A, 0.6))) and so forth. 

A formal semantics of GSMP is usually defined in terms of general state-space 
Markov chains (GSSMC, see, e.g., ||23]]). A GSSMC is a stochastic process <P over a 
measurable state-space (T, @) whose dynamics is determined by an initial measure u 
on (r,@) and a transition kernel P which specifies one-step transition probabilities^ 
A given GSMP induces a GSSMC whose state-space consists of all configurations, 
the initial measure p. is induced by a in a natural way, and the transition kernel is 
determined by the dynamics of GSMP described above. Formally, 

- r is the set of all configurations, and Q is a cr-field over F induced by the discrete 
topology over S and the Borel cr-field over the set of all valuations; 

- the initial measure p allows to start in configurations with zero valuation only, i.e. 
for A G Q we have p(A) = XseZemW a o( s ) where Zero(A) - {s e S \ (s, 0) e A}; 

- the transition kernel P(z,A) describing the probability to move in one step from 
a configuration z = (s, v) to any configuration in a set A is defined as follows. It 
suffices to consider A of the form {s'}xX where X is a measurable set of valuations. 
Let V and F be the sets of variable-delay and fixed-delay events, respectively, that 
are scheduled in s. Let F' c F be the set of fixed-delay events that can occur as first 
among the fixed-delay event enabled in z, i.e. that have in v the minimal remaining 
time u. Note that two variable-delay events occur simultaneously with probability 
zero. Hence, we consider all combinations of e e V and t e R>o stating that 

Z e ev JET Hit({e), t) • Win({e}, t)dt if F = 

Zeev£ Hit({e},f)- Win({e},t) dt + nH(F',u)-Wm(F',u) otherwise, 

where the term Hit(£', t) denotes the conditional probability of hitting A under the 
condition that E occurs at time t and the term Win(£', t) denotes the probability 
(density) of E occurring at time t . Formally, 

Hit(£,f) = Succ(s, £)(s') ■ l[v' e X] 

where l[v' e X] is the indicator function and v' is the valuation after the transition, 
i.e. v'(e) is ±, or v(e) + t, or for each old, or inherited, or new event e, respec- 
tively; and v'( a) = t . The most complicated part is the definition of Win(£', t) which 
intuitively corresponds to the probability that E is the set of events "winning" the 
competition among the events scheduled in s at time t. First, we define a "shifted" 
density function f e \ V ( e ) that takes into account that the time v(e) has already elapsed. 
Formally, for a variable-delay event e and any elapsed time v(e) < u e , we define 

f e (x + v(e)) 

fe\y(e)(x) = if X > 0. 

i (e) fe(y)dy 



P(z,A) 



2 Precisely, transition kernel is a function P : F x Q —* [0, 1] such that P(z, •) is a probability 
measure over (F, Q) for each z 6 F; and P(-,A) is a measurable function for each A e Q. 
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Otherwise, we define f e \ V (e)(x) = 0. The denominator scales the function so that 
f e \v{e) is again a density function. Finally, 



Win(£, f) 



J "CO 

, fcW)(y) dy if E = \e) c V 

Ticev X°° fc\v(c)(y) dy if E = F' c F 

otherwise. 



A run of the Markov chain is an infinite sequence cr = zo Z\ Zi- • • of configurations. 
The Markov chain is defined on the probability space (£?, y r , f) where Q is the set of 
all runs, T is the product cr-field (^) !=0 ^, and P is the unique probability measure such 
that for every finite sequence Aq, ■ ■ ■ , A„ e Q we have that 

n^oeA () , ■ • • , <2>„eA„) = J* ■ • • J M(*o) ■ P(zo, dzi) ■ ■ ■ P(z„-i , A„) 

where each <£, is the i-th projection of an element in i3 (the i-th configuration of a run). 

Finally, we define an m-step transition kernel P m inductively as P l {z,A) = P(z,A) 
and P M (z,A) = f P(z, dy) ■ P'(y,A). 



2.2 Frequency measures 

Our attention focuses on frequencies of a fixed state s € S in the runs of the Markov 
chain. Let cr = (*q, vo) (s\, vi) • • • be a run. We define 

d(o-) = hm — — c(cr) = hm — ^ — — 

«^oo n n^cc 2 i=0 V i+I (A) 

where 5{si) is equal to 1 when s, = s, and otherwise. We recall that V;+i(a) is the 
time spent in state Sj before moving to Sj+\. We say that the random variable d or c is 
well-defined for a run cr if the corresponding limit exists for cr. Then, d corresponds to 
the frequency of discrete visits to the state s and c corresponds to the ratio of time spent 
in the state s. 



2.3 Region graph 

In order to state the results in a simpler way, we introduce the region graph, a standard 
notion from the area of timed automata Q. It is a finite partition of the uncountable set 
of configurations. First, we define the region relation ~. For a, b e R, we say that a and 
b agree on integral part if int(a) = mi(b) and neither or both a, b are integers. Further, 
we set the bound B = max {{£ e , u e \ e e &} \ {°°}). Finally, we put (s\, vi) ~ ($2, v-i) if 

- si = s 2 ; 

- for all e e E(si) we have that v\{e) and vi(e) agree on integral parts or are both 
greater than B; 

- for all e,f e E(si) with v\{e) < B and Vi(/) < B we have that frac(vi(e)) < 
frac(v, (/)) iff frac(v 2 (e)) < frac(v 2 (/)). 
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Fig. 2. A GSMP of a producer-consumer system. The events p, t, and c model that a packet 
production, transport, and consumption is finished, respectively. Below each state label, there is 
the set of scheduled events. The fixed-delay events p and c have l p = u p = l c = u c = 1 and the 
uniformly distributed variable-delay event t has /, = and u t = 1. 

Note that ~ is an equivalence with finite index. The equivalence classes of ~ are called 
regions. We define a finite region graph G = ( V, E) where the set of vertices V is the set 
of regions and for every pair of regions R, R' there is an edge (R, R') e E iff P(z, R') > 
for some zeS. The construction is correct because all states in the same region have 
the same one-step qualitative behavior (for details, see Appendix lB.il ). 

3 Two fixed-delay events 

Now, we explain in more detail what problems can be caused by fixed-delay events. We 
start with an example of a GSMP with two fixed-delay events for which it is not true 
that the variables d and c are well-defined for almost all runs. Then we show some other 
examples of GSMP with fixed-delay events that disprove some results from literature. 
In the next section, we provide positive results when the number and type of fixed-delay 
events are limited. 

When the frequencies d and c are not well-defined 

In Figure [2] we show an example of a GSMP with two fixed-delay events and one 
variable-delay event for which it is not true that the variables d and c are well-defined 
for almost all runs. It models the following producer-consumer system. We use three 
components - a producer, a transporter and a consumer of packets. The components 
work in parallel but each component can process (i.e. produce, transport, or consume) 
at most one packet at a time. 

Consider the following time requirements: each packet production takes exactly 
1 time unit, each transport takes at most 1 time unit, and each consumption takes again 
exactly 1 time unit. As there are no limitations to block the producer, it is working for all 
the time and new packets are produced precisely each time unit. As the transport takes 
shorter time than the production, every new packet is immediately taken by the trans- 
porter and no buffer is needed at this place. When a packet arrives to the consumer, the 
consumption is started immediately if the consumer is waiting; otherwise, the packet 
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is stored into a buffer. When the consumption is finished and the buffer is empty, the 
consumer waits; otherwise, a new consumption starts immediately. 

In the GSMP in Figure |2j the consumer has two modules - one is in operation and 
the other idles at a time - when the consumer enters the waiting state, it switches the 
modules. The labels 1 and 2 denote which module of the consumer is in operation. 

One can easily observe that the consumer enters the waiting state (and switches the 
modules) if and only if the current transport takes more time than it has ever taken. 
As the transport time is bounded by 1, it gets harder and harder to break the record. 
As a result, the system stays in the current module on average for longer time than in 
the previous module. Therefore, due to the successively prolonging stays in the mod- 
ules, the frequencies for 1 -states and 2-states oscillate. For precise computations, see 
Appendix lA.il We conclude the above observation by the following theorem. 

Theorem 1. There is a GSMP (with two fixed-delay events and one variable-delay 
event) for which it is not true that the variables c and d are almost surely well-defined. 

Counterexamples 

In 15 4 1 there are algorithms for GSMP model checking based on the region construc- 
tion. They rely on two crucial statements of the papers: 

1 . Almost all runs end in some of the bottom strongly connected components (BSCC) 
of the region graph. 

2. Almost all runs entering a BSCC visit all regions of the component infinitely often. 

Both of these statements are true for finite state Markov chains. In the following, 
we show that neither of them has to be valid for region graphs of GSMP. 

Let us consider the GSMP depicted in Figure|3] This is a producer-consumer model 
similar to the previous example but we have only one module of the consumer here. 
Again, entering the state C-waiting indicates that the current transport takes more time 
than it has ever taken. In the state C-waiting, an additional event t' can occur and move 
the system into a state Sink. One can intuitively observe that we enter the state C-waiting 
less and less often and stay there for shorter and shorter time. Hence, the probability 
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that the event f occurs in the state C-waiting is decreasing during the run. For precise 
computations proving the following claim, see Appendix |A. 21 

Claim. The probability to reach Sink from Init is strictly less than 1 . 

The above claim directly implies the following theorem thus disproving statement 1 . 

Theorem 2. There is a GSMP (with two fixed-delay and two variable delay events) 
where the probability to reach any BSCC of the region graph is strictly smaller than 1. 

Now consider in Figure [3] a transition under the event p from the state Sink to the 
state Init instead of the self-loop. This turns the whole region graph into a single BSCC. 
We prove that the state Sink is almost surely visited only finitely often. Indeed, let 
p < 1 be the original probability to reach Sink guaranteed by the claim above. The 
probability to reach Sink from Sink again is also p as the only transition leading from 
Sink enters the initial configuration. Therefore, the probability to reach Sink infinitely 
often is limbec p" = 0. This proves the following theorem. Hence, the statement 2 
of 4 1 is disproved, as well. 

Theorem 3. There is a GSMP (with two fixed-delay and two variable delay events) 
with strongly connected region graph and with a region that is reached infinitely often 
with probability 0. 

4 Single-ticking GSMP 

First of all, motivated by the previous counterexamples, we identify the behavior of the 
fixed-delay events that may cause d and c to be undefined. The problem lies in fixed- 
delay events that can immediately schedule themselves whenever they occur; such an 
event can occur periodically like ticking of clocks. In the example of Figure[3] there are 
two such events p and c. The phase difference of their ticking gets smaller and smaller, 
causing the unstable behavior. 

For two fixed-delay events e and e', we say that e causes e' if there are states s, s' 
and a set of events E such that Succ(s, E)(s') >0, e e E, and e' is newly scheduled in s'. 

Definition 2. A GSMP is called single-ticking if either there is no fixed-delay event 
or there is a strict total order < on fixed-delay events with the least element e ( called 
ticking event) such that whenever f causes g then either f < g or f — g — e. 

From now on we restrict to single-ticking GSMP and prove our main positive result. 

Theorem 4. In single -ticking GSMP, the random variables d and c are well-defined 
for almost every run and admit only finitely many values. Precisely, almost every run 
reaches a BSCC of the region graph and for each BSCC B there are values d,c e [0, 1] 
such that d(cr) = d and c(<x) = cfor almost all runs cr that reach the BSCC B. 

The rest of this section is devoted to the proof of Theorem |4] First, we show that 
almost all runs end up trapped in some BSCC of the region graph. Second, we solve 
the problem while restricting to runs that start in a BSCC (as the initial part of a run 
outside of any BSCC is not relevant for the long run average behavior). We show that in 
a BSCC, the variables d and c are almost surely constant. The second part of the proof 
relies on several standard results from the theory of general state space Markov chains. 
Formally, the proof follows from Propositions Q~]and Instated below. 



10 



4.1 Reaching a BSCC 



Proposition 1. In single-ticking GSMP, almost every run reaches a BSCC of the region 
graph. 

The proof uses similar methods as the proof in [4 1. By definition, the process moves 
along the edges of the region graph. From every region, there is a minimal path through 
the region graph into a BSCC, let n be the maximal length of all such paths. Hence, in 
at most n steps the process reaches a BSCC with positive probability from any config- 
uration. Observe that if this probability was bounded from below, we would eventually 
reach a BSCC from any configuration almost surely. However, this probability can be 
arbitrarily small. Consider the following example with event e uniform on [0, 1] and 
event / uniform on [2, 3]. In an intuitive notation, let R be the region [0 < e < f < 1]. 
What is the probability that the event e occurs after the elapsed time of / reaches 1 
(i.e. that the region [e = 0; 1 < / < 2] is reached)? For a configuration in R with val- 
uation ((<?, 0.2), (/, 0.7)) the probability is 0.5 but for another configuration in R with 
((<?, 0.2), (/, 0.21)) it is only 0.01. Notice that the transition probabilities depend on the 
difference of the fractional values of the clocks, we call this difference separation. Ob- 
serve that in other situations, the separation of clocks from value also matters. 

Definition 3. Let 6 > 0. We say that a configuration (s, v) is (5-separated if for every 
x,y e {0} U \ v{e) \ e e E(s)}, we have either |frac(jc) - frac(y)| > 6 or frac(jc) = frac(y). 

We fix a 6 > 0. To finish the proof using the concept of <5-separation, we need two 
observations. First, from any configuration we reach in m steps a <5-separated configura- 
tion with probability at least q > 0. Second, the probability to reach a fixed region from 
any 5-separated configuration is bounded from below by some p > 0. By repeating 
the two observations ad infinitum, we reach some BSCC almost surely. Let us state the 
claims. For proofs, see Appendix IB. 21 

Lemma 1. There is 8 > 0, m s N and q > such that from every configuration we 
reach a 5-separated configuration in m steps with probability at least q. 

Lemma 2. For every 6 > and teN there is p > such that for any pair of regions R, 
R' connected by a path of length k and for any 5-separated z € R, we have P k (z, R') > p. 

Lemma|2]holds even for unrestricted GSMP. Notice that LemmaQ]does not. As in 
the example of Figure[3] the separation may be non-increasing for all runs. 

4.2 Frequency in a BSCC 

From now on, we deal with the bottom strongly connected components that are reached 
almost surely. Hence, we assume that the region graph G is strongly connected. We 
have to allow an arbitrary initial configuration zo = (s, v); in particular, v does not have 
to be a zero vector0 

3 Technically, the initial measure is yu(A) = 1 if zo 6 A and ji(A) = 0, otherwise. 
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Proposition 2. In a single-ticking GSMP with strongly connected region graph, there 
are values d,c € [0, 1] such that for any initial configuration Zo and for almost all runs 
a starting from zo, we have that d and C are well-defined and d(cr) = d and c(cr) = c. 

We assume that the region graph is aperiodic in the following sense. A period p 
of a graph G is the greatest common divisor of lengths of all cycles in G. The graph 
G is aperiodic if p = 1 . Under this assumption^, the chain is in some sense stable. 
Namely, (i) has a unique invariant measure that is independent of the initial measure 
and (ii) the strong law of large numbers (SLLN) holds for 0. 

First, we show that (i) and (ii) imply the proposition. Let us recall the notions. We 
say that a probability measure n on (T, Q) is invariant if for all A e Q 

n(A) = j n(dx)P(x,A). 

The SLLN states that if h : r — > R satisfies E K [h] < oo, then almost surely 

lim = E„[h], (1) 

n— >°o n 

where E n [h\ is the expected value of h according to the invariant measure n. 

We set h as follows. For a run (so, Vo)(si, vi) • • • , let h(<t>i) = 1 if s,- = s and 0, 
otherwise. We have E^h] < oo since h < 1. From (fl~|i we obtain that almost surely 

d = hm = E n [h\. 

As a result, d is well-defined and equals the constant value E n \h\ for almost all runs. 
We treat the variable c similarly. Let W{{s, v)) denote the expected waiting time of the 
GSMP in the configuration (s, v). We use a function t((s, v)) = W((s, v)) if s — s and 
0, otherwise. Since all the events have finite expectation, we have < oo and 

E„[t] < oo. Furthermore, we show in Appendix IB . 3 I that almost surely 

c = lim — = . 

»— 2"=i W(<Pi) E„[W] 

Therefore, c is well-defined and equals the constant E k [t]/E„[W] for almost all runs. 

Second, we prove (i) and (ii). A standard technique of general state space Markov 
chains (see, e.g., l23l ) yields (i) and (ii) for chains that satisfy the following condition. 
Roughly speaking, we search for a set of configurations C that is visited infinitely often 
and for some £ the measures P e (x, •) and P e (y, •) are very similar for any x,y e C. This 
is formalized by the following lemma. 

Lemma 3. There is a measurable set of configurations C such that 
1. there is k e N and a > such that for every z € r we have P k (z, C) > a, and 

4 If the region graph has period p > 1, we can employ the standard technique and decompose the 
region graph (and the Markov chain) into p aperiodic components. The results for individual 
components yield straightforwardly the results for the whole Markov chain, see, e.g., (9). 
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2. there is £ € N, /? > 0, and a probability measure k such that for every z € C and 
AeQwe have P c (z,A) > /3 ■ k(A). 

Proof (Sketch). Let e be the ticking event and R some reachable region where e is the 
event closest to its upper bound. We fix a sufficiently small 6 > and choose C to be the 
set of 5-separated configurations of R. We prove the first part of the lemma similarly to 
Lemmata Q] and [2] As regards the second part, we define the measure k uniformly on a 
hypercubeX of configurations (s, v) that have v(e) = and v(f) e (0, 6), for / + e. First, 
assume that e is the only fixed-delay event. We fix z — (s' , v') in R; let d — u e -v'(e) > 6 
be the time left in z before e occurs. For simplicity, we assume that each variable-delay 
events can occur after an arbitrary delay x e (d - 6, d). Precisely, that it can occur in 
an e-neighborhood of x with probability bounded from below by [} ■ s where ft is the 
minimal density value of all &. Note that the variable-delay events can be "placed" this 
way arbitrarily in (0, 6). Therefore, when e occurs, it has value and all variable-delay 
events can be in interval (0,6). In other words, we have P e (z,A) > (3 ■ k(A) for any 
measurable A c X and for I = |fi|. 

Allowing other fixed-delay events causes some trouble because a fixed-delay event 
f + e cannot be "placed" arbitrarily. In the total order <, the event / can cause 
only strictly greater fixed-delay events. The greatest fixed-delay event can cause only 
variable-delay events that can be finally "placed" arbitrarily as described above. □ 

5 Approximations 

In the previous section we have proved that in single-ticking GSMP, d and c are al- 
most surely well-defined and for almost all runs they attain only finitely many values 
d\ . . . ,dk and c\, . . . ,Ck, respectively. In this section we show how to approximate dfs 
and c,'s and the probabilities that d and c attain these values, respectively. 

Theorem 5. In a single-ticking GSMP, let d\, . . . , d^ and c\, . . . , q be the discrete and 
timed frequencies, respectively, corresponding to BSCCs of the region graph. For all 
1 < i < k, the numbers di and C, as well as the probabilities P(d — di) and P(c — c,) 
can be approximated up to any s > 0. 

Proof. Let X\, . . . ,Xk denote the sets of configurations in individual BSCCs and di and 
c, correspond to X,. Since we reach a BSCC almost surely, we have 

k k 

<P(d = dd = P(d = di | Reach(Xj)) ■ 9(Reach(X$) = V l[dj = d t ] ■ T(Reach(Xj)) 

7=1 7=1 

where the second equality follows from the fact that almost all runs in the y'-th BSCC 
yield the discrete frequency dj. Therefore, !P(d = d,) and di can be approximated as 
follows using the methods of [24|. 

Claim. Let X be a set of all configurations in a BSCC S, X$ c X the set of config- 
urations with state s, and d the frequency corresponding to < B. There are computable 
constants n \ , «2 £ N and p \ , p2 > such that for every i € N and zx 6 X we have 

\P(Reach(X)) - P(zo,X)\ < (1 - jPl ) U/ " lJ 
\d-P\zx,Xi)\ < (l-p2) liM 
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Further, we want to approximate c, = E„[r]/E n [W], where n is the invariant measure 
on Xi. In other words, we need to approximate J x T(x)n(dx) and J x W(x)n(dx). An n-th 
approximation w n of ^[W] can be gained by discretizing the part of the state space 
{(s, v) € r | Ve € E(s) : v(e) < n) into, e.g., 1/n-large hypercubes, where the invariant 
measure n is approximated using P" . This approximation converges to E„ [ W] since W is 
continuous and is finite. For the details of the following claim, see AppendixICl 

Claim. On each region, W is continuous, and is finite. 

This concludes the proof as t only differs from W in being identically zero on some 
regions; thus, E„[t] can be approximated analogously. 

6 Conclusions, future work 

We have studied long run average properties of generalized semi-Markov processes 
with both fixed-delay and variable-delay events. We have shown that two or more (un- 
restricted) fixed-delay events lead to considerable complications regarding stability of 
GSMP. In particular, we have shown that the frequency of states of a GSMP may not be 
well-defined and that bottom strongly connected components of the region graph may 
not be reachable with probability one. This leads to counterexamples disproving sev- 
eral results from literature. On the other hand, for single- ticking GSMP we have proved 
that the frequencies of states are well-defined for almost all runs. Moreover, we have 
shown that almost every run has one of finitely many possible frequencies that can be 
effectively approximated (together with their probabilities) up to a given error tolerance. 

In addition, the frequency measures can be easily extended into the mean payoff 
setting. Consider assigning real rewards to states. The mean payoff then corresponds to 
the frequency weighted by the rewards. 

Concerning future work, the main issue is efficiency of algorithms for computing 
performance measures for GSMP. We plan to work on both better analytical methods 
as well as practicable approaches to Monte Carlo simulation. One may also consider 
extensions of our positive results to controlled GSMP and games on GSMP. 
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A Details on counterexamples 

Definition 4. A distance of two events e and f (in this order) in a configuration (s, v) is 
frac(vCO-v(e)). 

A.l When the frequencies d and c are not well-defined: Proof of Theorem[T] 

In the following, we prove that in our example d and c are not well-defined for almost 
all runs. Namely, that there is a set of runs with positive measure such that for these 
runs the partial sums oscillate. 

After setting the initial distance of events p and c, every run stays in the 1 -states 
(labeled with 1) until the distance is lessened in the state 2 C-waiting. This sojourn in the 
1 -states is called the first phase. Then the run continues with the second phase now in the 
2-states until the distance is lessened again and it moves back to 1 -states and begins the 
third phase etc. Each phase consists of repeating several attempts, i.e. running through 
the cycle of length three. In each attempt the distance gets smaller with probability 
d (where d is the current distance) and stays the same with probability 1 - d due to 
the uniform distribution of t . This behavior corresponds to the geometric distribution. 
The density on the new distance is uniform on the whole d. A phase is called strong 
if the newly generated distance is at most half of the old one. Further, we define a 
half-life to be a maximum continuous sequence of phases where exactly the last one is 
strong. Every run can thus be uniquely decomposed into a sequence of half-lives. The 
random variable stating the distance at the beginning of the j-th phase of the 2-th half- 
life is denoted D, -j. Denoting the number of phases in the ;-th half-life by L(i) we get 
D n -\,L(i) - 2D„ i. Thus by induction, we have for all n, i e N and j < L(n - i), 

D„_ y > 2'' ■ D„,i (2) 

Further, let Su be the number of attempts in the j'-th phase of the 2-th half-life, i.e. a 
length of this phase. We can now prove the following lemma. Roughly speaking, there 
are runs (of overall positive measure) where some phase is longer than the overall length 
of all phases up to that point. Note that the precise statement of the lemma implies 
moreover that this happens even infinitely often on runs of overall positive measure. 

Lemma 4. There are a > and m > 0, such that for every n > 1 there is a set "R n of 
measure at least m of runs satisfying 

S„ t i > a ^ Sij 
i=l.j»-l 

Proof. We set a = 2/(3 • (6 + 2 ■ 3)) = 1/18 and m = 1/4 and let n > 1 be arbitrary. We 
define the set % n to be the set of all runs cr such that the following conditions hold: 

1. S„,i > l/(2D„,i), 

(the length of the "last" phase is above its expecation), 

2. for all 1 < 2 < n, L(i) < (n - i) + 3, 

(previous half-lives have no more phases than n + 2, n + 1, . . . , 5, 4, respectively), 
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3. for all 1 < i < n and 1 < j < L(i), S u < 3(n - i)/D u , 

(all phases in previous half-lives are short w.r.t their expectations). 

Denote D :- D„_i We firstly prove that S„ t \ > a 2i=i..«-i j=i..l(/) f° r ai l mns i n 
Due to the inequality (f2]i and requirements 2. and 3., we can bound the overall length of 
all previous phases by 

Zj " - Zj 2>-D ~ 2' ■ D ~ D ~ lab 

1=1. .n-l 1=1 1=1 

7=l.X(i) 

and conclude by the requirement 1 . 

It remains to prove that measure of %, is at least m. We investigate the measures 
of the runs described by requirements 1.-3. Firstly, the probability that 5„,i > is 
(1 - D„ i) 1 / 2D «. 1 , which approaches 1/ as « approaches infinity and is thus greater 
than 1 /2 for D nj i < 1 /2, i.e. for n > 2. Out of this set of runs of measure 1 /2 we need 
to cut off all runs that do not satisfy requirements 2. or 3. As for 2., the probability 
of i-th half-life failing to satisfy 2. is (l/2) (/w)+3 corresponding to at least (n — i) + 3 
successive non-strong phases. Therefore, 2. cuts off 2£i l/2 ( "~' )+3 = Ji/^i 1 l/2' +3 < 
l/2 !+3 = 1/2 3 . From the remaining runs we need to cut off all runs violating 3. 
Since the probability of each Sjj failing is (1 - Dy) 3 '"" y , the overall probability of 
all violating runs is due to [2] at most 

n-l L(i) n-l Un-i) n-l Un-i) 

X X" - d u ?^' d « -XX"- -w 3i/ZVy <X X"- 

i=l 7=1 1=1 7=1 1=1 7=1 

n-l OO 

< X(' +3X1- 2'D) 3//2 ' D < X(' + 3)d/e) 3 '' 
i=i i=i 

< 1/4 



(e 3 - l) 2 

Altogether the measure of H n is at least m = 1/2 - 1/8 - 1/4 = 1/8. □ 

Due to the previous lemma, moreover, there is a set *R of runs of positive measure 
such that each run of % is contained in infinitely many ??„'s. 

Let us measure the frequency of 1 -states (we slightly abuse the notation and denote 
by d(cr) and c(<r) the sum of frequencies of all 1-states instead of one single state s). 
We prove that neither d(cr) nor c(<x) is well-defined on any <x e %. Since attempts last 
for one time unit, non-existence of d(cr) implies non-existence of c(cr). Thus, assume 
for a contradiction that d(<r) is well-defined. Denote s,- the number of attempts in the 
z'-th phase. Because 1-states are visited exactly in odd phases, we have 

gLi Si ■ odd(i) 
d(cr) = hm — 

L i= i Si 

where odd(i) = 1 if i is odd and otherwise. By the definition of limit, for every s > 
there is no such that for all n > no 

Zfai Sj ■ odd(i) _ Sj ■ o dd(i) 



<e (3) 
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Due to the lemma, s„ > o2i=i„„-i *i happens for infinitely many both odd and even 
phases n on cr e %. Now let d(cr) < 1/2, the other case is handled symmetrically. Let 
e be such that a > rz^zgT^ i an d we choose an odd « > no satisfying s„ > a YJ£\ s i ^ 
si. Denoting A = i,- and (3 = ■ odd(z') we get from © that 



l-2£-d(tr) 



K) 



Q + S„ O + l-2 £ -d (t r) A Q (.) £ Ch 

A + i„ A ~ A + ,_ 2a ! d(tr) A A 1 - d(cr) - £ 

(**) £ 

> : : (l-d((T)-e) = £ 

1 - d(cr) - £ 



which is a contradiction with (|3). Notice that we omitted the absolute value from (01 
because for an odd n the term is non-negative. The equality (*) is a straightforward 
manipulation. In (**) we use, similarly to ©, that | j - d(<r)| < £. 



A.2 Counterexamples: Proof of Claim 

In the following, we prove that the probability to reach the state Sink is strictly less than 
1. 

Similarly as in the proof of Theorem Q] we introduce phases and half-lives and 
proceed with similar but somewhat simpler arguments. Let d be the distance of events p 
and c. Note that 1 —d is the maximum length of transportation so far. The initial distance 
is generated in the state C-waiting with a uniform distribution on (0, 1). After that, 
the distance gets smaller and smaller over the time (if we ignore the states where the 
distance is not defined) whenever we enter the state C-waiting. Each sequence between 
two successive visits of C-waiting on a run is called a phase of this run. After each 
phase the current distance is lessened. The density on the new distance is uniform on 
the whole d. A phase is called strong if the newly generated distance is at most half 
of the old one. Further, we define a half-life to be a maximum continuous sequence of 
phases where exactly the last one is strong. Every run can thus be uniquely decomposed 
into a sequence of half-lives (with the last segment being possibly infinite if C-waiting 
is never reached again). The random variable stating the distance at the beginning of the 
z'-th half-life is denoted by D,. By definition, D, < D,_i/2 and by induction, for every 
run with at least i half-lives 

A<l/2\ (4) 

Denoting the number of phases in the z'-th half-life by L(i), we can prove the following 
lemma. 

Lemma 5. There is m > such that for every n > 1 the set % n of runs cr satisfying 

1. cr does not visit Sink during the first n half-lives, and 

2. for every 1 < i < n not exceeding the number of half-lives of a, L(i)(o~) < 2 ■ i 
has measure at least m. 

This lemma concludes the proof, as there is a set of runs of measure at least m that never 
reach the state Sink. We now prove the lemma. 
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Firstly, for every n we bound the measure of runs satisfying the second condition. 
The probability that 2i consecutive phases are not strong, i.e. L(i) > 2i, is l/2 2i as t is 
distributed uniformly. Therefore, the probability that there is i < n with L(i) > 2i is less 
than 2" = i 1 /2 2 '- This probability is thus for all n e N less than ££i 1 /2 2i = 1/3. Hence, 
for each n at least 2/3 of runs satisfy the second condition. 

Secondly, we prove that at least m' of runs satifying the second condition also satisfy 
the first condition. This concludes the proof of the lemma as m' is independent of n (a 
precise computation reveals that m' > 0.009). 

Recall that D, < 1/2' and we assume that L(i) < 2i. Therefore, the probability that 
Sink is not reached during the z'-th half-life is at least (1 - 1 /2') 2 ' as f' is distributed uni- 
formly and the distance can only get smaller during the half-life. Hence, the probability 
that in none of the first n half-lives Sink is reached is at least 

n 

\\{\ - l/2') 2; 

!=1 

Thus, for every n, the probability is greater than n£i(l ~ 1/2') 2 ' =: m'. It remains to 
show that m' > 0. This is equivalent to m (l _ 1/2') 2 ' > -oo, which in turn can be 
rewritten as 

2 p ln {2Ti) < 00 

Since , 1 /i 2 converges, it is sufficient to prove that 

in(^) e 0(1,?). 

We get the result by rewriting the term in the form of an approximation of the derivative 
of In in 2' - 1 which is smaller than the derivative of In in 2' - 1 because In is concave 

□ 

B Proofs of Section H 

In this section, by saying value of an event e, we mean the fractional part frac(v(e)) 
when the valuation v is clear from context. Furthermore, by M we denote the sum of u e 
of all fixed-delay events. 

B.l Correctness of the region graph construction 

The correctness of the region graph construction is based on the fact that configurations 
in one region can qualitatively reach the same regions in one step. 

Lemma 6. Let z ~ z' be configurations and R be a region. We have P(z, R) > iff 
P(z',R)>0. 
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Proof. For the sake of contradiction, let us fix a region R and a pair of configurations 
z ~ z' such that P(z,R) > and P(z',R) = 0. Let z = (s, v) and z' = 0, /)■ 

First, let us deal with the fixed-delay events. Let us assume that the part of 
P(z,R) contributed by the variable-delay events V is zero, i.e. Y*eev f Q Hit({e},f) • 
Win({e), t) dt = 0. Then the set E of fixed-delay events scheduled with the minimal 
remaining time in z must be non-empty, i.e. some e e E. We have 



P(z,R) =Succ(s,E)(s')-l[v € R] -Y] U(o(y) dy > 
P{z',R) =Succ(s,E)(s')-l[v' eR]-Y\ fc\v(c)(y) dy = 

where s' is the control state of the region R and v and v' are the valuations after the 
transitions from z and z' , respectively. It is easy to see that from z ~ z' we get that 
v 6 R iff v' € R. Hence, P(z,R) and P(z',R) can only differ in the big product. Let us 
fix any c e V. We show that JT fc\v'(c)(y) dy is positive. Recall that the density function 
f c can qualitatively change only on integral values. Both z and z' have the same order 
of events' values. Hence, the integral is positive for V iff it is positive for v. We get 
P(z',R) > which is a contradiction. 

On the other hand, let us assume that there is a variable-delay event e e V such that 

Jr»oo /^oo 
Succ(s,{e})(s')-l[v,eR]-f lMe) (t)- ]~[ f cHc) (y)dydt>0 
c eV\W J < 

where v, is the valuation after the transition from z with waiting time t. There must be 
an interval / such that for every t e / we have that f e \ V (e)(t) is positive, l[v t € R] = 1, 

/>co 

and J f c \v(c)(y) dy > for any c e V \ {e}. From the definition of the region relation, 
this interval / corresponds to an interval between two adjacent events in v. Since z ~ z', 
there must be also an interval /' such that for every t e /' we have that fe\ V '(e)(t) is 
positive, l[v' t e R] = 1, and f c \v>(c)(y) dy > for any c e V \ {e}. Hence, P(z',R) > 0, 
contradiction. □ 



B.2 Proof of Proposition [T] 

Lemma [TJ There is 5 > 0, m e N and q > such that from every configuration we 
reach a ^-separated configuration in m steps with probability at least q. 

Proof. We divide the [0, 1] line segment into 3 • |fi| + 1 slots of equal length 5. Each value 
of a scheduled event lies in some slot. We show how to reach a configuration where the 
values are separated by empty slots. 

As the time flows, the values shift along the slots. When an event occurs, values of 
all the newly scheduled events are placed to 0. The variable-delay events can be easily 
separated if we guarantee that variable-delay events occur in an interval of time when 
the first and the last slots of the line segment are empty. 
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We let the already scheduled variable-delay events occur arbitrarily. For each newly 
scheduled variable-delay event we place a token at the end of an empty slot with its 
left and right neighbour slots empty as well (i.e. there is no clock's value nor any other 
token in these three slots). Such slot must always exist since there are more slots that 
3 ■ |£|. As the time flows we move the tokens along with the events' values. Whenever 
a token reaches 1 on the [0, 1] line segment, we do the following. If the valuation of 
its associated event is not between its lower and upper bound, we move the token to 
and wait one more time unit. Otherwise, we let the associated event occur from now 
up to time 5. Indeed, for any moment in this interval, the first and the last slots of the 
line segment are empty. The probability that all variable-delay events occur in these 
prescribed intervals is bounded from below because events' densities are bounded from 
below. 

The fixed-delay events cause more trouble because they occur at a fixed moment; 
possibly in an occupied slot. If a fixed-delay event always schedules itself (or there is 
a cycle of fixed-delay events that schedule each other), its value can never be separated 
from another such fixed-delay event. Therefore, we have limited ourselves to at most 
one ticking event e. Observe that every other event has its lifetime - the length of the 
chain of fixed-delay events that schedule each other. The lifetime of any fixed-delay 
event is obviously bounded by M which is the sum of delay of all fixed-delay events in 
the system. After time M, all the old non-ticking events "die", all the newly scheduled 
non-ticking events are separated because they are initially scheduled by a variable-delay 
event. Therefore, we let the variable-delay events occur as explained above for m steps 
such that it takes more than M time units in total. We set m = [M/6~\ since each step 
takes at least 6 time. □ 

Lemma [2} For every 6 > and k e N there is p > such that for any pair of regions R, 
R' connected by a path of length k and for any ^-separated z e R, we have P k (z, R') > p. 
Furthermore, P k (z,X) > p where X c R' is the set of (5/3*)-separated configurations. 

Proof. Let z e ^o, k e N, and Ro,Ri, ■ ■ - ,Rk be a path in the region graph to the region 
R = Rk- We can follow this path so that in each step we lose two thirds of the separation. 
At last, we reach a (5/3*)-separated configuration in the target region Rk- We get the 
overall bound on probabilities from bounds on every step. 

In each step either a variable-delay event or a set of fixed-delay events occur. Let 
5' be the separation in the current step. To follow the region path, a specified event 
must occur in an interval between two specified values which are ^'-separated. A fixed- 
delay event occurs in this interval for sure because it has been scheduled this way. For 
a variable-delay event, we divide this interval into thirds and let the event occur in 
the middle subinterval. This happens with a probability bounded from below because 
events' densities are bounded from below. Furthermore, to follow the path in the region 
graph, no other event can occurs sooner. Every other event has at least 5' /3 to its upper 
bound; the probability that it does not occur is again bounded from below. □ 

B.3 Proof of Proposition |2] 
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Proposition |2J In a single-ticking GSMP with strongly connected region graph, there 
are values d, c e [0, 1] such that for any initial configuration zo and for almost all runs 
<x starting from zo, we have that d and c are well-defined and d(<x) = d and c(cr) = c. 

Proof. First, we show using the following lemma that <t> has a unique invariant measure 
and that the Strong Law of Large Numbers holds for <t>. We prove the lemma later in 
this subsection. 

Lemma |3j There is a measurable set of configurations C such that 

1 . there is k e N and a > such that for every zefwe have P k (z, C) > a, and 

2. there is t e N, fJ > 0, and a probability measure k such that for every z e C and 
A € Q we have P c (z,A) > ft ■ k(A). 

A direct corollary of Lemma |3]is that the set of configurations is small. 

Definition 5. Let n e N, e > 0, and k be a probability measure on (T, Q). The set r is 
(n, e, /c)-small if for all z e r and A e Q we have that P n '{z,A) > s ■ v{A). 

Indeed, we can set n — k + t and e — a + /3 and we get the condition of the definition. 

Corollary 1. There is n e N, s > 0, and k such that T is (n, s, K)-small. 

From the fact that the whole state space of a Markov chain is small, we get the 
desired statement using standard results on Markov chains on general state space. We 
get that <P has a unique invariant measure n and that the SLLN holds for 0, see (9] 
Theorem 3.6]. 

From the SLLN, we directly get that d = E„[5}. Now we show that c = . Let 
us consider a run (so, vo) (si, Vi) ■ • • . By f,- we denote v,+i(a) - the time spent in the z'-th 
state. We have 



l im gk^ll^ = i im n = lim^^Lp 5( Si ) ■ tjjjn 



The fact that c(cr) is well-defined follows from the end which justifies the manipulations 
with the limits. It remains to explain the last equality. First, let is divide the space of 
configurations into a grid Cs- Each □ e Cs is a hypercube of configurations of unit 
length 6. By zu we denote the z'-th configuration of the run. We obtain 



The second limit equals by the SSLN to n{n). By taking 6 — > we get that (*) = £ W [W]. 



E„[W] 




22 



By similar arguments we also get that 

km = E„[t] 

n— ioo fi 

a 

For the proof of Lemma[3]we introduce several definitions and two auxiliary lemmata. 

Definition 6. A path (so, vq) • •■(s„, v n ) is 6-wide if for every < i < n the configuration 
(sj, v,) is 6-separated and for every < i < n any every bounded variable-delay event 
e e E(s,') we have v,(e) + V;+i(A) < u e — 5, i.e. no variable-delay event gets 6 close to its 
upper bound. 

We say that a path (so, Vo) • • • (s„, v„) has a trace sqE\ SiE\ • • • E„s„ if s, — S{ for 
every < ; < n and for every < i < n we can get from (si-\,Vi-\) to (*;, V,) via 
occurrence of the set of events E; after time v,(A). 

A path (so, vo) ■ ■ ■ (s n , v„) has a total time t ift = Y!J=\ v,(a). 

The idea is that a (5-wide path can be approximately followed with positive probabil- 
ity. Furthermore, as formalized by the next lemma, if we have different 6-wide paths to 
the same configuration z* that have the same length and the same trace, we have similar 
n-step behavior (on a set of states specified by some measure k). 

Lemma 7. For any 5 > 0, any n e N, any configuration (s n ,v„), and any trace 
soE\ ■ ■ ■ E„s„ there is a probability measure k and fj > such that the following holds. 
For every 6-wide path (so, Vo) • • ■ (■?«, v„) with trace T — soE\ ■ ■ ■ E n s„ and total time 
t > M and for every Y &@we have P"((so, v ), Y)>/3- k(Y). 

Proof. Recall that B - max({£,, u e \ e e £} \ oo). Notice that the assumptions on the 
events' densities imply that all delays' densities are bounded by some c > in the 
following sense. For every e 6 £ and for all x e [0, B], d(x) > c or equals 0. Similarly, 
f d(x)dx > c or equals 0. 

We will find a set of configurations Z "around" the state z„ = (s„,v„) and define 
the probability measure k on this set Z such that k(Z) = 1. Then we show for each 
measurable Y c Z the desired property. 

Intuitively, configurations around z n are of the form (*„, v') where each v'(e) is either 
exactly v(e) or in a small interval around v n (e). We now discuss which case applies to 
which event e for a fixed trace T. All the following notions are defined with respect to 
T. We say that the ticking event g is active until the i-th step if g e E(*o) n ■ ■ • n E(s,_i ). 
We say that an event e e E(*„) U E„ U { a) is originally scheduled in the i-th step by f if 

- either f - g and g is active until the i-th step or / is a variable-delay event; and 

- there is k > 1 and a chain of events e\ e E C] G E Ck such that e\ - f ,c\ = i, 
all e2,...,ek are fixed-delay events, occurence of each E c . newly schedules e, + i, 
occurence of E Ck newly schedules e, and e e E(s Ci ) n • • • n Ws n -2) n (E(*„_i) U { a}). 

Recall that the special valuation symbol A denoting the lenght of the last step is also 
part of the state space. Notice that in the previous definition, we treat A as an event that 
is scheduled only in the state s n -\. We say that the last step is variable if E„ is either a 
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singleton of a variable-delay event or all the events in E n are originally scheduled by a 
variable-delay event. Otherwise, we say that the last step is fixed. 

Intuitively, we cannot alter the value of an event e on the trace T (i.e., v'(e) = v(e)) 
if the last step is fixed and e is originally scheduled by the ticking event. In all other 
cases, the value of e can be altered such that V(e) lies in a small interval around v„(e). 
The rest of the proof is divided in two cases. 



The last step is fixed Let us divide the events e e E(s„) U {a} into three sets as follows 

e e A if e is originally scheduled by a variable-delay event and frac(v„(e)) + 0; 
e e B if e is originally scheduled by a variable-delay event and frac(v„(e)) = 0; 
e e C if e is originally scheduled by the ticking event. 

Let a\, . . . , ad be the disctinct fractional values of the events A in the valuation v„ or- 
dered increasingly by the step in which the corresponding events were originally sched- 
uled. This definition is correct because two events with the same fractional value must 
be originally scheduled by the same event in the same step. Furthermore, let F\ , . . . , Fd 
be the corresponding sets of events, i.e. frac(v„(e,)) = for any e, e F ( . We call a 
configuration z ~ z n such that all events e e (B U C) have the same value in z and z n a 
target configuration and treat it as a c/-dimensional vector describing the distinct values 
for the sets F\, . . . , Fd- A ^-neighborhood of a target configuration z is the set of con- 
figuration \z + C | C e (-6, 5) d ). Observe that the ^-neighborhood is a J-dimensional 
space. We set Z to be the (5/4)-neighborhood of z n ( the reason for dividing 8 by 4 is 
technical and will become clear in the course of this proof). Let Kd denote the standard 
Lebesgue measure on the if-dimensional affine space and set k{Y) := Kd(Y)/Kd(Z) for 
any any measurable Y c Z. 

In order to prove the probability bound for any measurable Y c Z, it suffices to prove 
it for the generators of Z, i.e. for <f -dimensional hypercubes centered around some state 
in Z. Let us fix an arbitrary z e Z and j < 6/4. We set Y to be the y-neighborhood of z- 
In the rest of the proof we will show how to reach the set Y from the initial state (s , v ) 
in n steps with high enough probability. 

We show it by altering the original £-wide path <j = (so, vq) ■ ■ ■ (s n , v n ). Let t\ , . . . , f„ 
be the waiting times such that f, = v,(a). In the first phase, we reach the fixed z 
instead of the configuration z„. We find waiting times t' v ...,t' n that induce a path 
cr' = (so, vq) (s\, v'j) ... (s n , v' n ) with trace T such that (s n , v' n ) = z and t\ = vJ(a). In the 
second phase, we define using cr' a set of paths to Y. We allow for intervals I\,...,I n 
such that for any choice t x € I\, . . . , t n € I n we get a path cr = (s Q , v ) (s i , vi) . . . (s„, v„) 
such that (s n , v„) e Y and f, = v,(a). From the size of the intervals for variable-delay 
events and from the bound on densities c we get the overall bound on probabilities. Let 
us start with the first step. 

Let vi, . . . , Vd be the distinct values of the target configuration z- Recall that |v,— a,-| < 
(5/4 for each Let r(l), . . . , r(d) be the indices such that all events in F t are originally 
scheduled in the step r(i). Notice that each is a singleton of a variable-delay event. 
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Fig. 4. Illustration of paths leading to the set Y. The original path cr is in the first phase altered 
to reach the target state z (its values vi,v 2 , and v 3 are depicted between a and cr'). In the second 
phase, a set of paths that reach Y is constructed by allowing imprecision in the waiting times - 
the transition times are randomly chosen inside the hatched areas. Notice that at most d smaller 
intervals of size y/2 can be used to get constant probability bound with respect to the size of the 
d-dimensional hypercube Y. Transitions with fixed-delay are omitted from the illustration (except 
for the last transition). 



As illustrated in Figure |4] we set for each 1 < i < m 



f; = < 



( e - v,_i(e) if e e Ej is fixed-delay, 

ti + ll^iih ~ Q + cij - vj if i = r( j) for 1 < j < d, 
U + Z' k Ji(tk - Q otherwise. 



Intuitively, we adjust the variable-delays in the steps preceding the original scheduling 
of sets F\,...,Fd whereas the remaining variable-delay steps are kept in sync with 
the original path cr. The absolute time of any transition in cr' (i.e. the position of a line 
depicting a configuration in Figure|4|i is not shifted by more than c5/4 since |v,-a,| < 6/4 
for any i. Thus, the difference of any two absolute times is not changed by more than 
6/2. This difference bounds the difference of |v,(e) - vj(e)| for any i and e e fi. Hence, 
cr' is (c)/2)-wide because cr is c5-wide. Furthermore, cr' goes through the same regions as 
cr and performs the same sequence of events scheduling. Building on that, the desired 
property z' n — z is easy to show. 

Next we allow imprecision in the waiting times of cr' so that we get a set of paths 
of measure linear in y d . In each step we compensate for the imprecision of the previous 
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Fig. 5. Illustration of construction of a for the empty set C and the last step variable. 



step. Formally, let T, denote t[ + 2t=i(*t - ft). For each 1 < i < m we contraint 



[r„ r,] if Ei are fixed-delay events, 

(r, -|, Ti+D if i = rfj) for 1 < j < d, 
(r,-f, r, + f) otherwise. 



The difference to <r' of any two absolute times is not changed by more than 6/2 
because the imprecision of any step is bounded by 6/4. Because cr' is (5/2)-wide, any 
path & goes through the same regions as cr'. The difference of the value of events in 
any F, in the state z n from the state z is at most y/2 because it is only influenced by the 
imprecision of the step preceding its original scheduling. Hence, z n £ Y. 

By v we denote the number of variable-delay singletons among E\,.. .,E„. From 
the definition of P, it is easy to prove by that 

p"(zo,y) > P n mm -(c-r) d -(c-6/2y- d > ( Pmm -c/2yy-6 n - d 

Since K d (Y) = (2 • y) d and K d {Z) = (2 • 6/4-) d , we have k(Y) = K d {Y)/K d {Z) = (4y/6) d . 
We get P n (zo, Y) > k(Y) ■ (6 ■ p m { n ■ c/8)" and conclude the proof of this case by setting 
s = (6- Pmm ■ c/8)". 



The last step is variable The rest of the proof proceeds in a similar fashion as pre- 
viously, we reuse the same notions and the same notation. We only redefine the differ- 
ences: the neighbourhood and the way the paths are altered. 

We call (s, v) ~ z n a target configuration if there is y e R such that for all events 
e e C we have v(e) - v„(e) = y and for all events e e B we have v(e) = v n (e). We 
set g — d + 1 if C is non-empty, and g — d, otherwise. We treat a target configuration 
as a g-dimensional vector describing the distinct values for the sets F\,...,F d and the 
value y, if necessary. Again, a (5-neighborhood of a target configuration z is the set of 
configuration {z + C \ C e (-6, 6) g }. We set Z to be the ((5/4)-neighborhoodof z n and set 
k(Y) :- K g (Y)/i<g(Z) for any any measurable Y c Z. We fix Y to be a y-neighborhood of 
a fixed z e Z. 
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The path cr' is obtained from the cr in the same way as before. We need to allow 
imprecision in the waiting times of cr' so that we get a set of paths of measure linear in 

<y8 



- For the case g = d + 1 it is straightforward as we make the last step also with 
imprecision +y/2. Precisely 



where E„ are originally scheduled in the m-th step if E„ are fixed-delay events, and 
m equals n, otherwise. The difference of the value of events in any F; in the state 
z n from the state z is at most y because it is influenced by the imprecision of the 
step preceding its original scheduling and also by the imprecision of the last step. 
Events in C have the difference of the value at most y/2 because of the last step. 
Hence, z n e Y. Again, we get that P"{zo, Y) > k(Y) ■ (6 ■ /? m ; n • c/8)" and conclude 
the proof by setting s = (6 ■ p m [ n - c/8)". 
- For the case g — d it is somewhat tricky since only at most d choices of waiting 
times can have their precision dependent on y. In each step we compensate for the 
imprecision of the previous step. Only the imprecision of the step preceding the 
first scheduling Ey is not compensated for. Otherwise, it would influence the value 
of events E\ in z„. Let Tf denote t' i + ^sjf k - h). As illustrated in Figure [5] we 
contraint 



The difference to cr' of any two absolute times is not changed by more than 3-6/6 = 
6/2 because the imprecision of any step is bounded by 6/6. Because cr' is (6/2)- 
wide, any path <x goes through the same regions as cr'. The difference of the value of 
events E\ in the state z„ from the state z is at most y/2 because it is only influenced 
by the imprecision of the last step. The difference of any other event e is at most 
2 ■ y/2 because it is influenced by the imprecision of the step preceding the original 
scheduling of e, as well. Hence, z„ € Y. 

Now, we get that P"(zo, Y) > k(Y) ■ (6 ■ p m ; n • c/\2) n and conclude the proof by 
setting s = (6 -p m i n -c/ 12)". □ 

Lemma 8. Let 6 > and R be a region such that the ticking event e is either not 
scheduled or has the greatest value among all events scheduled in R. There is n e N, 
6' > 0, a configuration z*, and a trace sqE\ ■ ■ ■ E„s n such that from any 6-separated 
Z € R, there is a 6' -wide path to z* with trace sqE\ ■ ■ ■ E„s n and total time t > M. 



ti e { 



(Tt - I T t + I) 
(T t - f , Ti + f ) 



if Ej are fixed-delay events, 

if i = r( j) for 1 < j < d or i = m, 

otherwise 



ti e • 
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Proof. We use a similar concept as in the proof of Lemma [T] Let us fix a ^-separated 
z e R. Let a be the greatest value of all event scheduled in z. Observe, that no value is 
in the interval (a, a + 6). When we build the <5-wide path step by step, we use a variable 
s denoting start of this interval of interest which flows with time. Before the first step, 
we have s :- a. After each step, which takes t time, we set s :- frac(s + f)- 

In the interval [s, s + 6] we make a grid of 3 • |£| + 1 points that we shift along with 
s, and set 5' — 6/(3 ■ |£| + 1). On this grid, a procedure similar to the 5-separation takes 
place. We build the £'-wide path by choosing sets of events E, to occur, waiting times f, 
of the individual transitions, and target states z, after each transition so that 

- every variable-delay event occurs exactly at an empty point of the grid (i.e. at a time 
when an empty point has value 0), and 

- the built path is "feasible", i.e. all the specified events can occur after the speci- 
fied waiting time, and upon each occurrence of a specified event we move to the 
specified target state with positive probability, 

These rules guarantee that the path we create is 6' -wide. Indeed, the initial configuration 
is 5-separated for 5 > 6 1 ', upon every new transition, the 5' -neighborhood of is empty, 
and every variable-delay event occurs at a point different form its current point, whence 
it occurs at least 5' prior to its upper bound. It is easy to see that such choices are 
possible since there are only S events, but 3 • \&\ + 1 points. 

Now we show that this procedure lasts only a fixed amount of steps before all the 
scheduled events lie on the grid. Notice that if the ticking event is scheduled in R, it lies 
at a point of the grid from the very beginning because we define the grid adjacent to its 
value. If it is not scheduled, it can get scheduled only by a variable-delay event which 
occurs already at a point of the grid. Values of any other scheduled fixed-delay event 
gets eventually placed at a point of a grid. Indeed, every such event gets scheduled by 
a variable-delay event next time, since we assume a single-ticking GSMP. We now that 
after time M, all the non-ticking fixed-delay events are either not scheduled or lie on 
the grid. Each step takes at least 5' time. In total, after n = [M/6'~\ + 1 steps with trace 
E\ , . . . , E„, we can set z* '■— z n - 

It remains to show that from any other ^'-separated configuration z' e R, we can 
build a (5'-wide path of length n, with trace E\,...,E n that ends in z* ■ We start in the 
same region. From the definition of the region relation and from the fact that all events 
occur in the empty interval (a, a + 6) we get the following. By appropriately adjusting 
the waiting times so that the events occur at the same points of the grid as before, we 
can follow the same trace and the same control states (going through the same regions) 
and build a path z' Q ■ ■ - z'„ such that z' n = Zn- Indeed, all scheduled events have the same 
value in z' n as in z„ because they lie on the same points of the grid. In fact, this holds for 
z'_j and Zn-i as well (because the first n - 1 steps take more than M time units) except 
for the value of A. Finally, also A has the same value in z' n as in z n because there is no 
need to alter the waiting time in the last step. By the same arguments as before, the built 
path is also (5'-wide. □ 

Lemma |5J There is a measurable set of configurations C such that 
1 . there is k € N and a > such that for every zefwe have P k (z, C) > a, and 
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2. there is I e N, [5 > 0, and a probability measure k such that for every z e C and 
A € £ we have P f (z, A) > £ • k{A). 



Proof. We choose some reachable region R such that the ticking event e is either not 
scheduled in R or e has the greatest fractional part among all the scheduled events. 
There clearly is such a region. We fix a sufficiently small 8 > and choose C to be the 
set of ^-separated configurations in R. Now, we show how we fix this 8. 

It is a standard result from the theory of Markov chains, see e.g. [25, Lemma 8.3.9], 
that in every ergodic Markov chain there is n such that between any two states there 
is a path of length exactly n. The same result holds for the aperiodic region graph G. 
From Lemma [1] we reach in m steps from any z e r a (^'-separated configuration z' 
with probability at least q. From z\ we have a path of length n to the region R. From 
Lemma [2] we have p > such that we reach R from z' in « steps with probability at 
least p. Furthermore, we end up in a (<J'/3' 1 )-separated configuration of the region R. 
Hence, we set 8 = 8' 13" and obtain the first part of the lemma. 

The second part of the lemma is directly by connecting Lemmata[8]and[7] 

C Proof of Theorem |5] 

It only remains to prove the two Claims. 

Claim. Let X be a set of all configurations in a BSCC S, X$ c X the set of config- 
urations with state s, and d the frequency corresponding to < B. There are computable 
constants n\, n% e N and p\,pi > such that for every i e N and zx £ X we have 

\P{Reach{X)) - P\zo,X)\ < {l-pif M 
\d-P(z x ,X s )\ < (l-p 2 ) yM 

Proof. Let Y denote the union of regions from which the BSCC S is reachable. By 
Lemmata Q] and [2] we have p,q > and m € N and k < |V] such that from any z e Y 
we reach X in m + k steps with probability at least p ■ q. We get the first part by setting 
n\ = m+k and p\ - p-q. Indeed, if the process stays in Y after n\ steps, it has the same 
chance to reach X again, if the process reaches X, it never leaves it, and if the process 
reaches f\(lU Y), it has no chance to reach X any more. 

By Corollary [T]in Appendix IB. 3 1 r is (n, e, /e)-small. By Theorem 8 of [24] we thus 
obtain that for all x e r and all i € N, 

sup|f"'0,A)-7!-(A)| < (l-e) L// " J 

Aeg 

which yields the second part by setting A = {(s, v) e r \ s — s) and observing d - n{A) 
and A e Q □ 

Claim. On each region, W is continuous, and £^[W] is finite. 
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Proof. Let (s, v) be a configuration, C and D the set of variable-delay and fixed-delay 
events scheduled in s, respectively. For nonempty D let T = min^^ - v{d)) be the 
time the first fixed-delay event can occur; for D — we set T = 00. The probability that 
the transition from (s, v) occurs within time t is 



as non-occurrences of variable-delay events are mutually independent. Observe that 
F(t) is piece-wise differentiable on the interval (0, T), we denote by /(f) its piece-wise 
derivative. The expected waiting time in (s, v) is 



Recall that for each variable-delay event e, the density f e is continuous and bounded as 
it is defined on a closed interval. Therefore, f e \ t are also continuous, hence F and / are 
also continuous with respect to v and with respect to t on (0, T). Thus W is continuous 
for T both finite and infinite. Moreover, for finite T, W is bounded by T which is for 
any (s, v) smaller than max^ £ g {j. Hence, is finite. For T = 00, Zs^W] is finite 

due to the assumption that each f e has finite expected value. □ 





for T < 00, 
for T = 00. 



(5) 
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